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\ SCALE FOR EVALUATION OF PUPILS’ JUDGMENTS ON 
BEST-LIKED AND LEAST-LIKED ASPECTS OF 
SECONDARY SCHOOLS 
KENNETH W. EELLS 
Statistician, Cooperative Study of Secondary School Standards, 
Washington, D. C. 

THE Cooperative Study of Secondary School Standards has been en- 
ved during the past year in an intensive study of all phases of the work 
nd equipment of 200 earefully selected secondary schools in all parts 
the country. These schools were chosen on a proportional basis so as 
take into consideration the following features: geographical distribu- 
on, control, enrollment, racial groups served, form of organization, type 
community served, type of program offered, sex of pupils enrolled, 
nominational control, and boarding or day school status. Of the entire 
oup, 168 are publie schools, 32 are private; 175 are accredited, 25 are 
ot thus recognized. The number of schools per state varies from one to 

velve 

As part of the program for evaluating these schools over 20,000 of 
eir present and former pupils were asked to indicate their judgment 

on various aspects of their school’s program. As one phase of this study, 
was desired also to obtain a general judgment concerning the entire 

ool on the part of each of these pupils. This might have been done by 

ny one of several techniques.-First, the pupils might have been asked 
0 indicate on a linear seale the degree of their general satisfaction with 
their school experience. Second, they might have been given a list of 
significant items characterizing a good school and have been asked to 
eck those which they felt were present, or absent, in their own school. 
Third, they might have been asked to state in their own words what they 
ced best and liked least about their school. 

The advantages of the first method, use of a linear seale of degree of 
tisfaction, are that it permits expression in a single judgment of the 
titude of the pupil and such pupil judgments may be easily summarized 

tatistically. The chief disadvantages are that it is very general, and that 
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it perhaps calls for a perspective possessed by but very few secondary 
school pupils, most of whom know no other schools with which to com 
pare their own. The advantages of the second method, a check-list o 
good features of a school, are that it suggests points to be considered 
which might otherwise have been forgotten, and that judgments ex 
pressed by pupils may be summarized by comparatively simple statis 
tical procedures. Its main disadvantage is that it directs the pupil’s 
thinking into the channels suggested by the compiler of the checklist. TI 

advantage of the third method, unguided indication of the best-like 

and least-liked features of the school, is that it measures the school ; 

both extremes rather than only at the middle or average point, and th 

it is a completely undirected evaluation on the part of the pupil. Th 
chief disadvantage is the difficulty involved in summarizing in a usefu 
form the subjective replies of a large number of pupils when given i 
this form. If this disadvantage can be satisfactorily overcome, the 


method should prove of decided value to administrators and other ir 


vestigators in evaluating the judgments of pupils and graduates and 

perhaps even of staff members, parents, patrons, and other groups. 
The ¢ ooperative Study has used the first and third of these methods 
: It is the purpose of this article to present the seale which was develope: 
to summarize the unguided responses of approximately 17,300 present 


pupils and of nearly 7,000 former pupils to the following two questions 


‘What do vou like best about your school yee 
‘*What do you like least about your school ?’’ 


rABULATION OF THE REPLIES 


Upon inspection of the responses, it was apparent that they could be 
classified, without doing serious violence to the pupils’ meanings, int 
about 250 different classes. These responses were grouped into twelve 


main classes and 42 sub-classes. The main groups were as follows: 


a. The school staff 
b. The pupil population 
ce. The pupil activity program 
: d. The curriculum and courses of study 
e. Instruction 
f. Administration 
g. Outcomes of the school program 
h. Plant and equipment 
1. Library 


j. Guidance 
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k. Miscellaneous 
l. Non-evaluative re plies 


The sub-divisions and individual items, as finally revised, appear in 
Seale which is ineluded as part of this article. The replies were coded 
punching on Hollerith tabulating ecards. After a preliminary sum- 

of the frequencies of each of the replies, those items with fewer 


five replies were combined with other related items or put into the 


llaneous classifications. 


EVALUATION OF THE REPLIES 


In order to evaluate these replies, even when classified, it was neces 

to arrive at some determination of which items were of greater, and 

1 were of le sser. significance as judgments or eriticisms of the school. 
accomplish this, the entire list was sent to twenty-three educators 
nected with the Coéperative Study. This group was composed of 
en members of the Executive Committee of the Study, eight members 
the Field Committees which had just completed a year of visiting and 
dying the 200 coéperating schools, and three members of the office 


of the Study. 


lhe instructions accompanying the list were as follows: 


(in the attached page is a elassified list ot the r sponst S of over 20 000 
esent and former high school pupils to the qui stions “Wi at do you 
lo you like least about your 


liked best, 


e best about your school’’ and ‘‘ What 
hool?’’ The items on the left side of the pages are thos 
ile those on the right side are those liked least. Most of the items oecur 
pairs, some pupils liking the feature and others disliking it. 
You are asked to indicate in the blanks in the middle of each page 
ir best judgment as to the importance of each item as an eva 


the school. In assigning weights to the different items, please use the 


ollowing five point seale: 


extremely important 


» 

/—very important 
i—of average importance 
2—somewhat important 


1—of little or no importance 


Seve n types ot responses rect ived which do not seem to hy usable as 


+ 


luations of the school have been placed at the end of the list, under 


Non-evaluative answers.’’ If there are other items which you feel rep 
sent neither praise of a school nor reflection upon it, mark them with 


oy” 





n 


aspect 


ll 


CSS 
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general, it 1s assumed that the ‘‘best-liked’’ and ‘‘least-like 


of each item will receive the same weighting. If, however. vou { 


i Some cases that the receognition ot the presence ota feature IS more, 


Important than recognition of its absence, you may mark the ite 


4 


with two weights. The first will apply to the ‘‘best-liked’’ aspect of t 
item and the second to the ‘‘ least liked’’ aspect. 
In assigning these weights, please keep in mind the following « 
siderations 
l We are trying to secure a valid method of evaluating the 
schools, not the pupils 
2. Weight the items for their importance as indications of the 
qualitv of the school as a whole: Cz... & pupil’s indication 
that the best feature of a school is the curriculum as a whole 
probably represents greater praise of a school than a similar 
indication that the best feature is a particular English course 
Consider how much the pupil is likely to know about the 
item; e.g., a pupil’s comment on pupil activities is probably 
more reliable than one about the school board 
t+. Consider the likelihood of the pupil’s giving a particular an 
swer. For example, it is to be expected that a large number of 
supils will think of sports or other activities when asked what 
is the best feature of the school. On the other hand, if a pupil 
says that the library, or the guidance program, is the best 
thing t is like ly to indicate that that feature is rather out- 
ndings 
'wenty-one replies were received out of the twenty-three requested 
Althoug the J ulges had been invited to assign different weights 1 
tne vest -| ed iIspect na least liked’’ aspect o1 inv response if the 
felt that such a difference was desirable, only eight judges used this typ: 
( wi ti The was only o fem To W ch as many as three judg 
ale ible r 
\s to be « " d. the twentv-one judges differed somewhat 1! 
( ulgments, the means of their weights varying from 1.9 for one judg 
| oO Te another T e mean tor the entire group was 2.9. Just slight] 
helow the point defined as ‘‘of average importance.’’ The mean weig! 
for fifteen of the twenty-one judges fell within 0.4 of this value. or be 
tween 2.5 and 3.5 
The means of the weights assigned to each item by the three groups 
0 i\dges—Exeeutive Committee, Field Committee, office staff—wer 
comp | separately, but were found not to differ markedly from eae 
other on most items. The maximum difference between the group « 
idges assigning highest ratings and the group assigning lowest ratings 
oO rn singl tem was 1.7 but only elgnt yx reent of the items had dit 
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ces in exeess of one full point between the groups of judges giving 
est and lowest ratings. The average difference was 0.6. 
‘he means of the weights assigned by the entire group of judges on 
of the 225 items ranged from 1.3 to 4.3, with a mean of 2.9. The 
dard deviation of these mean weights was 0.7. The mean weights 
arranged in order and separated into five classes, at intervals of 
standard deviation, beginning one-half standard deviation on either 
of the mean. This gives a distribution which is very close to that 


en by the normal probability curve: 


Class 
lean Weichts Actual Normal Value 
) and 6.2¢ 6.7% 5 
2 3 29 25.3% 24.2% ‘ 
1.6 2.2 17.3% +, eae ; 

l l ~4.5 4.2 
9 and under 6.7% 6.7% 1 
l ] 


lv, class values ranging from one to five were assigned to the five 
sses, as indicated in the last column. 


\ class value of ‘‘O’’ was assigned to the seven ‘‘Non-evaluative’”’ 


esponses which appear at the end of the scale. Although the judges were 


ited to indicate additional items that should be considered as useless 
‘ evaluation purposes, only five of the twenty-one judges made any 

indication ; on only two of the 225 responses evaluated did as many 
three judges agree that it should be considered as non-evaluative. 
ese votes were counted as weights of ‘‘O’’ in computing the mean 


ohts for each item. 


RELIABILITY OF THE CODING AND THE SCALI 


The usefulness of the seale as derived depends, of course, upon its re 
bility and the reliability of the factors comprising it. 

Considerable judgment is involved in classifying some of the original 
sponses of the pupils into the 225 items listed on the seale. A state 
ent by the pupil that the best feature of the school is ‘‘ personal in 
rest of the teachers in the pupil’’ might seem to one person to belong 
der Code No. 042, (See following Seale) as ‘‘ Qualities of the teachers: 
sight and understanding’’ and to another, or to the same person at 
other time, to belong under Code No. 432, as ‘‘ Methods of teaching: 
dividual attention from teachers.’’ To test the reliability of this trans 
tion of the pupil’s original response into a code number, a typical 


hool’s responses were coded three times: (1) by a person familiar with 


e code, (2) by the same person, six weeks later; (3) by a second per- 





— 





eleven mean weights ranged from .09 to .15, with an average of .12. Sin 
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son, entire ly unfamiliar with the code. A total of 197 items were 1 


es inde pe nae ntly Of these, 84 percent were coded ix 
ically In each of the three codings: only one pereent were classifies 
ree differ: nt ways by three codings. 

Between the two codings by the same person, six weeks apart, 92.4 
ereent were coded identically, and in only 5.6 pereent of the e 

e discrepancy great enough to shift the pupil’s responses f1 
one subdivision of the seale to another. In only 1.5 percent of the ec: 


did the response shift Irom one main elassifieation to another. It wo 


seem. the retore that the translation of free subjective responses into 
objective classification of this type can be accomplished with a satisf 
To! at eree OL Ye lj ib lity 


Be tween two codings. one by L person who had used The code in 0 


24 000 eases ind the othe r by a person who had never seen the eode 


e, 86.8 percent of the items were classified identically. Differences 
classification were great enough in 8.1 pereent of the cases to move t 
tem from one subdivision to another, and in 5.1 percent of the cases t 
shift the item into a different main group. This would indicate t! \ 
while it is better for the code and seale to be applied by someone fami! 
tT, the pl neipal of a school who has never seen the seale can usé 
wit! reasonable expectation of obtaining comparable results. 


The reliabilitv of the five elass val 1es assigned to the differs nt iter 


s indicated by the probabl errors of each of the main weights. Thes 


vere not computed for all 225 items, but were computed for eleven 


em, selected at half standard-deviation intervals from the mean, so 
ye TY il of the entire range of responses The probablk errors of thes 


class interva ised in assigning the elass values was .66, or five ar " 
lf times the average probable error of the means, the chances of 
mean weight varying by as much as a class interval are co the ¢ 
siderably less than one in a thousand, (.0002). Since any mean weig tw 
vould ve to shift at least one class interval, and usually more, to chang | 
S | weight by more than one point, the chances of this oecurren 
etically negligible. Of course the final weight for certain iten , 
might ry by one point, in eases in which the placement of items 
near the boundaries between the five classes used for the seale. 


USE OF THE SCALE 





{ ooperative st id Seale, as it appears at the end of this artic 


includes a Hollerith code number for each item. which will be found eo 








ie 
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whether the data is to be analyzed by machine or by hand. The 
ligit of each number represents the twelve main classes into which 
ms are grouped: All the 000’s, for instance, have to do with the 
staff, the 100’s with pupil population, ete. The second digit rep- 
ts the subdivision to which the item belongs; the 020’s all repre- 
eachers in general, the 030’s, 040’s and 050’s represent qualities of 
whers. The third digit identifies the particular response. If Hol- 
machine analy sis is used, the class value ean be punched in the 
eolumn of the ecard. This plan makes available a maximum of 
mation with a minimum of effort, especially where a large number 
ses are being analyzed. Sorting, whether by machine or by hand, of 
rst digit gives a tabulation of the frequency with which the main 
are mentioned. For certain purposes this may be all that is de- 
Sorting on the second digit similarly will give a tabulation by the 
visions. On the other hand, if a final score only is desired, without 
tabulation of frequencies of particular answers, sorting by the class 
s alone is sufficient. 
\ school seore is derived by analyzing completely the best-liked fea 
es, then repeating the process for the least-liked measures. Each answer 
e question as to the best-liked feature in school carries a positive 
t equal to the class value of the item mentioned. Likewise an answer 
e question as to the least-liked feature carries a negative weight equal 
e class value of the item mentioned. In this way two pupils who list 
features of the school, one as best-liked and one as least-liked, which 
of equal importance as indicated by their class values, have no net 
on the school seore. A school can accumulate a postive score only 
ts best features, as seen by its pupils, are more important than its 
rest features. The total positive secore—which is, of course, the total 
the class values assigned to the answers of the first question—minus 


total negative score gives a net score for the school. If comparisons 


tween schools or between groups of schools are to be made, this score 


id be divided by the number of pupils answering to allow for the 
erence in number of pupil responses in the different schools. The 
ult will be a figure indicating the average evaluation of a school by 
pupils, as derived from their judgments as to the best-liked and least 
d features of the school. 


An application of this seale to the 200 coop rating schools of the Co 


rative Study, including norms for different types of schools, will be 


sented after analy sis of the data has been completed 
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COOPERATIVE STUDY SCALE FOR EVALUATION OF RESPONSES 0 
PUPILS AS TO BEST-LIKED AND LEAST-LIKED 
FEATURE OF SCHOOL 
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\ EXPERIMENTAL STUDY OF THE VALUE OF INFORMAL 
METHODS IN TEACHING PRIMARY READING 


LESTER R. WHEELER 


; 


State Teachers Colle ae, Johnson ¢ ily, T: NnESSle 


OnE of the major objectives of the new curriculum is to make use of 
situations so that the child not only gains experience in living, but 
motivated by the necessity to learn. The merits of this new program 
ve been discussed widely by various educators, and there appear ob- 
us advantages in placing teaching emphasis on problems having social 
lues rather than on isolated facts. Play activities in the form of *‘ pro}- 
ts’’ commonly serve as the medium of instruction in the more progres- 
e schools. Many teachers are using various types of games which serve 
s good teaching devices: Woody (28) suggests Dominoes, Parchesi and 
Lotto as good number experiences in the lower grades; and Bingo, Old 
Maids, and Anagrams are used in some schools. Hooper and Stratton (12 
find Ring Toss, Bean Bag, Peggity, Peter Rabbit, Jack Straws, Winnie- 
the-Pooh, Tom Mix Cireus, ete., suecessful in grades I and Il. Since these 
re rames are part of the child’s actual environment they furnish excellent 
material for school experiences ; they are, however, limited in teaching 

ossibilities and are not easily integrated into the various phases of the 


new curriculum. 

During the past year there has appeared a series of games especially 
adapted to the requirements for supplementary material that teaches 
through play * In order to determine the efficiency of these games as well 
s to make an objective analysis of the value of informal activities in 
icademice instruction, we have undertaken several experiments. The fol 
lowing discussion outlines an experiment to improve reading through the 


ise of edueational games. 

The READ-O games are built from Gates’ Vocabulary List for Pri- 
mary Grades arranged throughout the series according to frequency ; 
[A contains the 72 words most commonly used, IB the 72 next most com- 


mon, IC the next 72. ete., so that the whole series uses 432 words from 


the first 500 in Gates’ list. As some of the 500 are duplicated because of 





Gates’ arrangement according to the different parts of speech, the 432 
different words in the READ-O games cover practically all of Gates’ first 
READ-O, Series IA and IB, first half Grade 1 
READ.-O, Series IC and ID, second half Grade 1 


READ-O, Series ITA and IIB, Grade I] 
Augsburg Publishing Co., Morristown, Tennesse: 


aor 




















f JOURNAL OF EDUCATIONAL RESEARCH [| Vol. 31, N 


five hundred. Series LA, consisting of the first 72 words most frequen 
used, was used for this experiment. 

The game is based on a flasheard technique. The teacher presents « 
flasheard at a time, calls the word, and uses it in a sentence taken fr 
the context of the children’s reading material. Each child plays on 
individual ecard which has sixteen of the 72 words arranged in fo 
vertical and horizontal rows. No two playing ecards are alike and t \ 
72 words are distributed equally over the forty-five children’s cards. As 
the teacher presents a flasheard, each child looks over the sixteen words 
on his eard, and if he finds the word which has been presented, he put 
a token over that word. The first child to get four tokens in a row eal 
out “‘READ-O.’’ The teacher and pupils then check the child for a 
curacy by having him read the four words in the row he has completed 
If they cheek with the flasheards which are displayed in the flashear 
holder, the child wins the game. Through the game the children learn 
recognition and meaning of words, and also phrase, clause and sentenc 
building, all of which may be closely associated with their other reading 
materials. Because of the large degree of active participation by eacl 
child in the group, the constant frequency in the repetition of the words 
and the general construction of the game, READ-O readily lends itselt 
to experimentation both for studying the learning processes and for for 
mulating word lists based on difficulty. 

Data for this experiment were gathered during twenty consecutiv 
school days in October and November, 1936, from 227 children who had 
entered grade IB the previous September. Six different publie schools ~ 
in various sections of Johnson City, Tennessee, and the Training School 
of the Teachers College were selected in an attempt to get a fairly ae 
curate sampling of schools in this vicinity. Seven teachers, with four 
years college training and several years experience, were chosen as rep- 
resenting the average publie school teacher in our city systems. The 
classes were divided into experimental and control groups according to 


intelligence rating on Dearborn IA Tests. Where a teacher had two see 





tions, one was put into the control group and one into the experimental 
so as to equalize as much as possible any differences due to personalities 
and particular methods of the teachers. Also the experimental and con- 
trol groups were equally divided between afternoon and morning sessions 
so that time of day would not influence the final results. Beeause we 1 
wanted to determine the value of this play technique when used by the 


average teacher in the average school, no attempt was made to supervise : 
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sist the teachers in any way other than instructing them as to the 
il procedure of playing the game with the experimental groups. In 
xperimental group READ-O was played twenty minutes a day dur- 
art of the regular reading period. Otherwise both groups received 
same type of instruction. The codperation of all the teachers was 

le nt 
\s there are no adequate standardized vocabulary tests available to 
first grade so early in the school year, and as we wanted an accurate 
on the particular words used in the game, the best method of test- 
ppeared to be oral recognition of words presented individually to 
hild. The validity of this method of testing words out of context has 
established by Monroe (16) and it has been used by several investi- 
rs. All the children in the experimental and control groups were 
ted in this manner at the beginning of the experiment, at the end of 
days and at the end of twenty days. The test consisted of the 72 words 
he flasheards for READ-O IA, presented one at a time. Interest was 
tained by making a game of it, giving the child all the ecards he eould 
mediately recognize. All the testing was done by students especially 
ned for this method. At the end of the ex perime nt the Devault Stand- 
ed Reading Test, Form I, was given to all the children to determine 
relative progress of both groups in general reading ability. Every 
caution was taken to equalize the conditions of both groups through- 
the experiment. All instruction was done by the teachers under 
rmal publie school conditions. Possibly the first grade in the Training 
school was somewhat superior in intelligence and general advantages 
n the average publie school; for this reason it was put into the control 
Table I shows the results of Tests I, II and III. Both groups are nor- 
| in intelligenee, with a slight advantage in favor of the control group ; 
P.E. indicates there is no significant statistical difference. Test | 
ws that at the beginning of the experiment the experimental group 
s slightly superior in sight vocabulary even though slightly handi 
yped in ability as measured by intelligence rating. This discrepancy 
vht be explained by the fact that about 30 percent of the control group 
nsisted of children in the Training School, above the average in intelli 
nee but receiving less formal instruction during the first month of 
ool than is usually given in the public schools. As the formal methods 


e introduced in the Training School after the first month or six weeks 


school, the method should not materially affect the final results 
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\fter ten days, the experimental group showed a wider spread 
greater gall the children playing READ-O gained from 5.63 to | 
words, while the control group gained from 2.56 to 5.66 words. Su 
vains were show! by the first and third quartiles The experim 
group made a greater gain during the last ten days of the experime 
nereasing e median seore from 10.56 to 21.6. while the control © 
reached 9.44 or less than half the gain made by the experimental gr 
Eve f we should assume that there is a difference between the 
ero s in rate of learnine because possibly of different methods ot 
nine nd t the rate Is three words less as indicated in Test | 
experimental group would still show about twice as much gain as 
control group. As the experiment progressed the differences betwee 


Tecrease 


Group 


Tt 
between thi ldren who played READ-O and those taught by the 1 
ir school methods. The percentage of overlapping also shows the 
erence increases in favor of the experimental group. There were tw 
s many children in the control group who reached or exceeded 
median of the experimental group at the beginning as there were at 
end o e experiment 
Comparing the vocabulary gain in Table II and Figure I give 
clearer picture of the two groups. The median difference in the gain 
e experimental and control groups during the first ten days is abe 
fwo words ile the difference is over seven words at the end of the s 
ond ten days ver the entire twenty day period the difference is nea 
en words, 01 er two and one-half. times as great as the gain for 
ntrol gro This is verified by a study of the P.E.’s, the reliability 
e differences, and further substantiated by a study of the quartiles ar 
ercentage of overlapping. With an inerease in the number of days 
ra etic he « ference between thy groups increases 1n favor of the ¢ 
perimental group: the longer the children play READ-O the ereater t 
difference becomes. The teachers in the experimental group observed t! 
experiment progressed, interest in the game also increased 
rABLE Ill 
\ GA L) 
: a. ' 
“4 
( 4 ] ‘ 
21 6 


d in favor of the experimental group. A study of 


differences indicates a significant statistical differs 
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lable III shows the daily gain of the children in the experimental 
i control groups. During the first ten days the experimental! group 
ed an average of .55 or slightly over one-half a word a day, while 
control group gained about one-third of a word a day. For the entire 
enty days, the experimental group made an average gain of .84 of a 

rd a day, while the control group gained .34 each day. The control 


up gained less during the second ten days of the experiment than 
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during the first ten days. In view of the fact that these first 72 words are 
fairly common in the reading vocabularies of pre-primers and primers, 
it would seem that the ordinary methods of instruction do not furnish 
enough repetitions of these words to insure permanent learning and im. 
mediate recognition. More words were forgotten from one testing to the 
next by the control than by the experimental group, indicating that fre 
quency of use is an important factor in learning the basie primary vo- 
eabulary. Other investigators have pointed out that to get the highest 
retention, new words should be reviewed within twenty-four hours after 
presentation (4). 

Another picture of the vocabulary gain of the two groups is given in 
Figure II. Here the ogive, or cumulative graph, illustrates the frequency 
distribution of the experimental and control groups. The number of cases 
making no gain is less in the experimental than in the control group 
about 6 percent of the children made no gain in the experimental group 
while 23 percent in the control group failed to make any gain. Q-1 ir 
the experimental group equals and slightly surpasses the median of th 
control group, i.e., the lower 25 percent of the experimental group mad 
as great a gain as the lower half of the control group. The median gait 
of the experimental group was around 16 words compared with 12 words 
gained by 75 percent of the control group. Q-3 or 75 percent of the ex 
perimental group equaled the gain made by approximately 95 percent of 
the control group. The difference in range is also shown by these graphs, 
indicated by the length of the curves. The greatest gain made by any 
child in the experimental group was 64 words compared with a maximum 
of 38 in the control group. Approximately 10 percent of the experimental 
group exceeded the greatest gain made by the control group. 


TABLE IV 


COMPARISON OF EXPERIMENTAL AND CONTROL GROUPS ON THE DEVAULT READING TES 


Median No Median 

I Q Cases Range Score P.E Q-1 Q-3 Q 
Experimental 98 + 927 116 0-28 16.53 546 11.00 20.40 4 
( t 1 1 16 64 ( 2° 1 504 80 13.40 s 
Difference 2 + 1.77 6.03 806 5.20 7.00 

Db 

P.Evaire 1.13 7.48 
Chances in 1 77 100 


Overlapping 10 
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[f a child gains in word recognition, does this help his general reading 
ability? The teachers of the experimental group observed that interest 
transferred from READ-O to other reading activities and also that there 
followed a marked improvement in general reading ability. At the end 
of twenty days the Devault Primary Reading Test, Form I, was given to 
both groups. Table IV and Figure I show a median score of 16.53 for the 
experimental group and 10.5 for the control group; a significant statisti 
cal difference of 6.03 in favor of the experimental group. A similar di! 
ference is indicated by the first and third quartiles. A study of the over- 
lapping of the groups shows that only 10 percent of the control group 
reached or exceeded the median of the experimental group. The relation- 
ship of word recognition learned in the experiment to reading ability is 
further shown by the high positive correlations between the Devault Read 


ing test seores and the vocabulary test scores: 


Devault Reading test scores and vocabulary test | r= 51+ .05 
Devault Reading test scores and voeabulary test I] r= .63 = .04 


Devault Reading test scores and vocabulary test IIT r= .64 = .04 


An analysis of the words used in the Devault Reading tests shows that 
only thirty-six, or 50 percent, of the words used in the game of READ-O 
occur in Devault’s test. If there is a high positive correlation with onl) 
half the words common to both, an increase of the commonality of words 
would give a still higher and more significant correlation between learn 
ing words through the game and ability to read as measured by the 
Devault Reading test. The results indicate a definite relationship be- 
tween recognition of words and achievement in reading,—an important 
factor for primary teachers to consider. 

In conclusion it seems that the fundamental value of the game tech 
nique used in this experiment lies in the fact that READ-O trains the 
pupil in habits of perception more effectively and with greater retention 
than the formal methods used by the average class-room teacher. The 
data indicate that a free-type method of informal instruction, when 
closely integrated with the curriculum and constructed so as to conform 
to the psychological principles of economical learning, definitely increases 


the rate of learning. 
SUMMARY 


1. The majority of teachers »gree that play is valuable for motivation, 
and present tendencies in the new curriculum indicate a wider interest in 
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more practical uses made of free-play activities as the basis for in- 
ial methods of instruction. 

». A number of the games used by classroom teachers are designed 
rely for entertainment and are limited in practical teaching possi- 
Les. 

}. This experiment was conducted to test the merit of informal meth- 
; of instruction and the efficiency of educational games for the im- 
vement of primary reading. 

!. The results show that children who play the game of READ-O as 
supplementary reading activity in the classroom show greater vocab- 
iry development than children taught by regular classroom methods. 

5. There seems to be a high correlation between word recognition as 

rned in the READ-O game and general achievement in reading as 
easured by standardized reading tests. 

6. It appears that the methods used by the average classroom teacher 

» not furnish adequate repetitions of the subject matter to insure per- 
nanent learning and immediate reeall. 

7. The results of this experiment indicate that informal methods based 

n free-play activities may be successfully used in the classroom for in- 
structional purposes, and, when closely integrated into the curriculum, 


definitely increase the rate of learning. 
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ACCURACY IN TESTING THE ARTICULATION 
OF SPEECH SOUNDS 
FLORENCE M. HENDERSON 
University of Hawaii 

FREQUENTLY observations of speech-sound articulation have been re- 
ported, and more and more the data obtained are being treated statis. 
tically. In such studies, articulation tests are prepared for the purpose 
and administered to the subjects by an examiner who records the re- 
sponses. In some instances, recordings are made by letter spellings and 
in others by phonetic symbols. The former method, fortunately, is fast 
falling into the diseard, and at present no study using this procedur 
can be seriously treated. However, the phonetic transcription of 1 
sponses, significant as this advance is, does not solve all of the problems 
involved in securing accurate and measurable data in the field of speech- 
sound articulation, for it is at once evident that the results of such studies 
are subject to errors in the examiner's judgment of the responses and to 
inconsistency of response on the part of the subjeets; i.e., individual 
subjects may vary their responses on successive administrations of th 
test. 

It seems that tests of speech-sound articulation should conform as 
nearly as possible to the requirements for other objective tests, such as 
intelligenee and achievement tests; i.e., they should show satisfactory 
validity, objectivity, and reliability. However, articulation tests differ 
somewhat from these other tests for several reasons. First of all, th 
purpose (or purposes) in giving articulation tests differ to some extent 
from the purpose in giving either intelligence or achievement tests. hh 
tests of speech-sound articulation, one is interested not only in a subject's 
relative rating on a numerical seale, as is the case in most other tests, but 
also in the exact types of errors made. In fact the latter objective is 
usually the more important of the two. Second, no definite objectiv: 
standards for judging thé responses can be set up beeause there is no 
well-defined limit as to what is right or wrong, since the recording of th 
responses Is entirely dependent upon personal judgment. Third, there 
is as yet no practical way to secure a permanent record of the speech 
judged, and as a result it is difficult to check the extent of a given ex- 
aminer’s personal bias after a study has been completed. These facts 


as will be shown in the following paragraphs, make it necessary to em 


ploy special techniques for determining the acceptability of articulation 
test results. 
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retofore, in studies of the articulation of speech-sounds investiga- 
eem for the most part to have contented themselves with finding 
efficients of reliability, apparently assuming that such coefficients, 
ficiently high, indicate consistency in the responses of the subject 
iate to establish the validity of the result, and consistency and ob- 
ity in the examiner’s judgments sufficient to guarantee the ac- 
ey of the findings. This, however, is not true as will be shown. 
f an articulation test is to be valid, the exact articulatory produc- 
s, as recorded by the examiner, should represent a given subject’s 
stomary articulation of the sounds tested. Let us suppose that an ex- 
er who is perfectly aceurate in his judgments of speech sounds 
ts the test word bathe and records a substitution of (v) for the final 
sound. Now suppose he administers the test again and elicits the sub- 
tution (d) for the th sound. The coefficient of reliability will not be 
‘ted by this change in response, but it certainly would not be a valid 
nelusion to report that the customary substitution is (v). If such 


variation in responses were limited to only a few subjects, the conelu- 


us in terms of exact substitutions would not be greatly affected, but 
| great number of subjects were to manifest marked inconsistency of 


response. definite conclusions as to the substitutions made would not be 


lid. This point was well illustrated in a recent study made by the 
ter.2. The coefficient of reliability (Pearson product-moment) of a 
st given on two successive occasions to sixty institutionalized children 
om three years, six months of age through the second grade was 

01 whereas that for the same test when administered to sixty in- 
tutionalized children from the third to the seventh grade, inclusive, 
s 92 + .02. Since this difference was contrary to expectation, the re- 
ective pairs of records were compared for consistency of response, 


ich procedure revealed that, although the scores were relatively con- 


tant. the exact errors noted varied mueh more for the smaller children 


in for the older ones. Sinee the examiner had previously shown by 


Poole furnishes an exception to this statement. She secured a statistical measure 
includes both her consistency in judgment and the consistency of the subject’s 
onses. It seems to the writer that these two measures should be kept, separate, for 
is no reason why a well-qualified examiner should not be allowed to proceed with 
tudy even though an investigation of two suecessive sets of responses to the same 
ny the same subjects should reveal a high decree of inconsistency in the records. 
haps this very inconsistency on the part of the subjects is a significant finding. 
Llrene. The Genetic Deve lopment of Consonant Sounds in Children’s Speech. 
iblished doctor ’s dissertation. University of Michigan, 1954. 


{ St uly of the {rticulation oT Consonants hy Norn al Institi tionalized ( hildren. 


iblished doetor’s dissertation. University of Wisconsin, 1935. 
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experiment a rather high degree of consistency in judgment, this con 
dition was probably due to the fact that the smaller children produced 
so few sounds correctly that the scores remained the same although th 
responses were changed. In such an instance, the validity of the test in 
terms of exact articulatory productions is so low that no definite con. 
clusions are permissible except that the speech-sound patterns are, i 
general, very poorly established. 

Suppose we take for granted, then, that any examiner will find th 
reliability of his test and attempt to determine the validity insofar as 
this is dependent upon consistency of the individual subject’s responses 
Two questions still remain: (1) Is such variation as may be noted }y 
tween responses on two successive administrations of the same test to 
the same subject by the same examiner attributable to inconsistency of 
response or to inconsistency in the examiner’s judgment? (2) Are th: 
responses that are recorded the actual responses given, i. e., would other 
trained phoneticians agree with the judgments made by the examiner 






The first question is related to the matter of validity, for if variations 






in the recorded responses are attributable to inconsistency in the ex- 






aminer’s judgments, a conclusion that the subjects are exceedingly vari 






able in their responses would be erroneous. The first consideration 






preparing to give a test, therefore, would seem to be the establishment 






of a satisfactory consistency in the examiner’s responses. The second 






question has to do with the objectivity of the test. Since, as pointed out 






above, there are no completely objective standards for judging the r 






sponses and sinee it is impossible to check the extent of an examiner’s 






personal bias after a study has been completed, there seems to be no way 






to determine that a test is objective except to establish by experiment 






before a given problem is undertaken that the prospective examiner is 






satisfactorily accurate in his judgments, i. e., that he agrees with other 






well-trained phoneticians both as to the correctness or ineorrectness of 






the responses and to the exact productions in the responses judged to 






be ineorrect. 







In the writer’s mind, then, four statistical measures should be es 
tablished in connection with every test of speech-sound articulation bé 






fore the results of such a test can be considered satisfactorily accurate 






These measures are, in order of their determination: (1) the degree oi 






consistency of the examiner’s judgments, (2) the degree of objectivity 






in the examiner’s judgments, (3) the extent of consistency of respons 






on the part of the subjects, and (4) the coefficient of reliability of the 








ment 


the 


OTISES 


ed hy 








a 





1938] ACCURACY IN TESTING SPEECH SOUNDS 351 





est. Pertinent facts gleaned from experience in determining these meas- 
will be reported here. 
onsistency of judgment can be established through the use of an 
ical recording of the test words as produced by a subject who mani- 
ts markedly defective articulation. Using the same phonograph and 
cord form that provides for a separate record of each test unit, the 
spective examiner may judge the reproduced speech on two succes- 
oceasions separated by a considerable interval of time. The record 
m may be prepared like one of the following, as illustrated by the use 
e stimulus word ‘‘eat’’ in a test of both vowels and consonants: 
1) eat , or (2) . When the two sets of judgments have been 
i t 
red, they ean be compared, item for item, and the percentage of 
cement ean be computed. The pereentage thus secured would be the 
ex of consistency of judgment as exhibited by the examiner. Since in 
| an experiment all variables exeept the Judge are reduced to a mini- 
m. it seems reasonable to demand a rather high percentage of agree- 
ment before an examiner be allowed to undertake a research problem. In 
- study of the speech of institutionalized children® the writer mani- 
fested a consistency of judgment on 98.2 percent of the items, and a co- 
jperating judge showed a self-agreement of 90.2 percent. A small amount 
research would quickly establish a standard that would be acceptable 
the pursuit of research studies. 
The establishment of objectivity in the examiner’s judgment is a 
ch more complicated process. Since in the above-mentioned study 
he writer’s index of consistency of judgment seemed rather high, she 
proceeded to discover how accurate she was in her estimates of articu- 
tory productions. Lacking any well-established technique with which 
to compare her findings, she undertook a meager study on the general 
subject of objectivity of speech sounds. A discussion of the procedure 
nd findings is submitted here with the hope of stimulating more and 
tter controlled research on the subject. 
The study was initiated to determine to what extent several phonet- 
lly trained judges would agree upon the articulation of consonants as 
oduced by subjects with markedly defective articulation. In pursuing 
e desired information, three different approaches were used. The same 
st words, involving one hundred sixty-seven consonant units, were used 


Op. cit., p. 30. 
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in all three procedures. The words were so chosen as to permit each of 
twenty-three consonants to appear at least twice in each position—init 
medial, and final—in which they appear in the English language. 

In the first experiment the subjects to be tested appeared in person 
before the judges. Two third-grade children who exhibited unusual ab 
normalities of articulation were given the test by the writer in the pres 
ence of two other phonetically trained judges. The writer, who also a 


as a judge, elicited the test words by the presentation of objects, 


tures, and parts of the body to be named, questions to be answered, and 
a few words to be repeated after her. Each of the three judges ind 
pendently recorded the articulatory responses on a form which provided 
for a separate record of the production of each consonant unit. Insofar 
as possible each response was transcribed phonetically and omissions 
were indieated by a dash. The three records were then compared in 
order to determine the extent to which the three judges agreed with on 
another. 

Speech over a microphone afforded the material used for the seeond 
experiment. A five-year old child from the speech clinie at the Unive: 
sity of Wisconsin was given the test by an assistant. The two were seated 
by themselves before a microphone, and the three judges who served in 
the first experiment received the amplified speech in another room wher: 
they judged it as in Experiment I. The results were analyzed just as 
in the first experiment. 

As a third experiment an electrical recording of the test words as 
they were produced by one of the coéperating judges was used. In mak 
ing the record the speaker quite deliberately distorted both consonants 
and vowels, sometimes using sounds for which there are no definit: 
phonetic expressions. The record was then judged separately by th 
writer and the second cooperating judge, and the two sets of judgments 
were compared and evaluated as in the first two experiments. 

The results of the three different experiments were analyzed fron 
two different points of view; viz., (1) agreement among judges in terms 
of the correctness and incorrectness of the responses, and (2) agreement 
among judges in terms of the exact recordings of responses. The former 
method takes into account those sounds that were judged to be incorrectly 
articulated and ignores the problem of exactly what each judge thought 
the subjects produced, whereas the latter plan involves the judgments 
of exactly what sounds were produced. The results of these analyses 


are given in Table I. 
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TABLE I 
PERCENTAGES OF TEST UNITS ON WHICH THE VARIOUS JUDGES AGREED 
Agreement among Three Agreement between Respec- 
Judges tive Pairs of Judges Disagreement 
c ‘ co 2 2 of All Judges 
ment Correctness & Exact Correctness & Exact as to Exact 
Incorrectness Responses Incorrectness Responses Responses 
Speech of @ 14&2...86 12...79 
Subject 1&3...88 1&3...80 
rsor 80 72 243...87 24&3...81 6 
Spe h over 1 & 2 84 1 & 2 .69 
1&3 78 1&3 75 
rophone 69 60 2&3 77 2&3 75 10 
Ii! 
trically 
rded aa —_ 143...90 143...85 15 


Although the numbers of subjects and judges used in the study are 
too small to warrant any very definite conclusions, a study of the data 
sented above makes certain statements possible: 

1. Correetness and incorrectness of consonants can be judged more ac- 
eurately than the exact articulatory productions themselves. 

2. If seventy-five percent agreement, which is the commonly accepted 
criterion of objectivity in ratings of items whose evaluation depends 
upon personal judgment, be taken as a standard, correctness and in- 


correetness of consonants as articulated by subjects in person and in 
recorded speech possess a considerable degree of objectivity (a) when 
the judges have good hearing and (b) when the judges are practiced 
and expert in the detection of detailed speech sounds. 

Fine distinctions of consonants, under many circumstances, possess a 
rather low degree of objectivity, but when the material to be judged 
is speech of a subject in person or speech electrically recorded, the 
articulatory productions are satisfactorily objective when judged by 
well-qualified persons. 

}- Consonants transmitted over a microphone appear to possess a low 
degree of objectivity even from the point of view of correctness and in- 
correctness. 

To those who have never questioned the objectivity of speech sounds, 
e above table may be a little surprising. However, when we consider 
ie personal variables that may limit the extent of the judges’ agree- 
ent, such as (1) the auditory sensitivity for certain frequencies in the 
peech range, (2) phonetic background, (3) experience in articulation 


testing, and (4) choice of phonetic expressions to express slightly non- 
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ar’ 


Ja 


standard speech sounds, the results of these experiments are not remark 
able.* Since all of these variables are more or less controllable, it seems 
that some effort should be made to reduce their influence to a minimun 
before any study of objectivity of speech sounds is made. Unfortunately, 
with the exception of testing hearing, these factors were not taken int 
consideration in making the experiment reported here. 

Relative to future studies of this type that may be undertaken, th 
following suggestions are made: (1) By audiometric test each judg 
should manifest practically normal hearing before he is qualified ty 
serve in the experiment. (2) Important differences between the phonetir 
backgrounds of the various judges should be reduced to a minimum by 
the use of a training period in which the judges independently trans. 
eribe the responses of several subjects and later discuss the symbols em. 
ployed. (3) Each experimenter should participate in testing situations 
until he can make judgments with ease and can demonstrate consider- 
able consistency in his judgments from one occasion to the next. (4) 
Certain modes of expressing recurrent non-standard articulatory pro- 
ductions should be adopted by mutual agreement during the training 
period. 

It is regrettable that no extensive research has been done relative to 


igatol 
ertall 


objectivity in speech-sound articulation testing, for lacking well-estab- C 
lished standards as to the extent of agreement that one might expectfJ,,. a, 
when several well-trained persons concurrently judge another’s articu- FR ..,... 
lation, an individual desirous of testing his own objectivity in judging }§ ;),. , 
speech sounds is unable to inverpret fully whatever data he may obtain jecti 
from an experiment such as these presented here. The writer, therefore, J} +), oy, 
recommends further that more extensive and better controlled research f§ yy) ;; 
be done with two aims: first, to establish the best method for determining § yjoy 
such objectivity ; and, second, to set up standards with which future inves- FJ men 
*In addition to these personal restrictions to agreement between judges, eachP§ of r 
method has factors inherent in itself which make for limited agreement. For instance, c 
in experiments I and II the subjects were often asked to repeat a word in order that Pie 
the judges might more accurately grasp which sounds were being used. Since the 9% tim 
subjects were a little confused by being asked to repeat what seemed very plain to}® ¢,.., 
them, they often varied the productions during the repetitions. This situation fre- 
quently resulted in one judge’s choosing one production for his final record while sho’ 
another judge hit upon some other. Experiments IT and IIT have certain disadvantages @ ..},; 
not found in I; they are (1) the elimination of visible articulatory movements and 
(2) the distortion of frequency and intensity composition of the various articulatory |@ CON 
productions. Such a reduction and masking of cues forces one to do a certain amount @ +}, 
of guessing, and it is not probable that any two judges will guess the same in «ll 
instances. These uncontrollable factors make for differences between the extent of @ °'l 
agreement in the several experiments and operate to determine the relative value of the @ .f 


technique to be used in future studies. 
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igators in the field may compare their measures. In making such a study 
ertain procedures would seem to be warranted; viz., (1) the use of more 
than three judges, (2) the use of the same subjects and the same test 
vords for all experiments, (3) the qualification of all judges as to hear- 
ing ability, phonetic training, experience in testing including satisfac- 
tory consistency in judgment, and (4) the adoption of some scheme for 
expressing recurrent non-standard articulatory responses. 

Onee a prospective examiner is provided with a well-established 
standard of objectivity in articulation testing, he can arrange his own 
experiment under conditions similar to those used in securing the stand- 
ard. In this way the percentage of agreement among the judges can be 
ascertained, and a comparison can be made between the judgments of 
each individual judge and those of the remainder of the group. Marked 
disagreement on the part of any individual judge should lead to a fur- 
ther study of the accuracy of his judgment, possibly to be followed by a 
repetition of the experiment. When the experiment is finally finished, 
the examiner can compare with the standard his percentage of agree- 
ment with the other judges in order to learn whether he is sufficiently 
accurate in his judgments to accept the responsibility of gathering data 
from which significant conclusions may be drawn. 

Consistency of response can be determined by giving the same test 
to an adequate sample of unselected subjects—say one hundred—on two 
successive oceasions not too widely separated in time. For this purpose 
the record form used in establishing the examiner ’s consistency and ob- 
jectivity in judgment should be used. The two sets of responses should 
then be compared and the percentage of the total number of items on 
which there is agreement should be computed. If the examiner has pre- 
viously shown a high consistency in judgment, the percentage of agree- 
ment found in this experiment may be considered the index of consistency 


| articu. 
judging 
’ obtain 
erefore, 
‘eseare) 
‘mining 
e inves- 


es, each P§ of response by the subjects. 

oan Since in any articulation test, each test item is usually presented several 
ince the}} times, it will be necessary to use a second record form to find the scores 
_—* from which the coefficient of reliability is found. The type of record used 
‘d while) should provide for a composite of responses for each speech sound used 
ep which will readily yield to scoring. The results of the tests given for 
eulatory | consistency of response can be transferred to this new record form for 
oo the purpose of scoring. The process is a bit cumbersome, but where many 
tent of § crrors in articulation are made, it seems the only answer to the problem 
® of the | of determining both validity and reliability. Once the scores are ob- 
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tained, the coefficient of reliability may be computed by one of the well. 
known methods. 


1. 


2re 


SUMMARY 


Tests of the articulation of speech sounds are subject to errors in 
the examiner’s judgment and to errors resulting from inconsistency 
of response on the part of the subjects. 

Since, frequently, the chief object in giving an articulation test is to 
determine the exact types of errors made, it is necessary to employ 
certain peculiar techniques in order to guarantee the validity and ob- 
jectivity of test results. 


. In order to establish satisfactory accuracy in the results of a test, 


the writer suggests that four statistical measures be determined: (a) 
consistency of the examiner’s judgments, (b) objectivity in the ex- 
aminer’s judgments, (¢c) consistency of the subjects’ responses, and 

(d) the coefficient of reliability. The first and third measures are re- 

lated to validity, and the second measure is basie to the objectivity 

of a test. 

Detailed techniques are suggested for the determination of the first 

three measures. 

A limited study on the general subject of the objectivity of speech 

sounds is reported. From the data obtained, the following statement 

may be made, pending further study: 

a. Correctness and incorrectness of consonants as articulated by sub- 
jects in person and in recorded speech possess a considerable de- 
gree of objectivity (1) when the judges have good hearing and 
(2) when the judges are practiced and expect in the detection of 
detailed speech sounds. 

b. Fine distinctions of consonants, under many circumstances, possess 
a wey low degree of objectivity, but when the material to be 
judged is speech of a subject in person or speech electrically re- 
corded, the articulatory productions are satisfactorily objective 
when judged by well-qualified persons. 

e. Consonants transmitted over a microphone appear to possess a 
low degree of objectivity even from the point of view of correctness 
and incorrectness. 

It is recommended that extensive and well-controlled research be in- 

itiated by way of (a) determining the best technique for establishing 

objectivity in judgment and (b) setting up standards of consistency 
and objectivity in judgment. 
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SOME ASSUMED DETERMINERS OF TEACHERS’ ATTITUDES 


GrorGE W. HARTMANN 
Teachers College, Columbia University 


THE sources of an individual’s point of view on controversial, phil- 
osophie, educational and social issues have been variously sought in con- 
stitutional peculiarities of temperament, the accidents of personal ex- 
periences and the pressures exercised by organized groups upon their 
members. A deterministic psychology has not favored the notion that 
people’s opinions about unresolved human problems have generally been 
reached by careful preliminary examination of all available evidence 
with the final judgment the inevitable outcome of the interplay of facts 
and relations. Instead it is held that an attitude—whether it be a broad 
general orientation or a specific response to a limited situation—is a fune- 
tion of the nature, history and affiliations of the organism, and conse- 
quently causally correlated with such factors. 

An approach akin to that just described was used in organizing and 
interpreting the results of a national survey of the social attitudes and 
information of American secondary school teachers.’ Assuming that we 
have a defensible measure? of the liberalism-conservatism of a given 
group of teachers, what fluctuations in this variable occur when instrue- 
tors are classified according to other discriminable traits? Most of these 
assumed ‘‘correlates’’ may prove to be relatively unimportant influences, 
but it is only by searching along a fairly wide front that the truly decisive 
‘‘forees’’ ean possibly be revealed. 

It seemed desirable to determine what effect, if any, the different sub- 


*For a description of the tests used and a discussion of the basic findings, see 
Chapter 8 of The Teacher and Society, the first Yearbook of The John Dewey Society 
for the Study of Education and Culture, edited by W. H. Kilpatrick and published by 
Appleton-Century, 1937. The data in this paper rest upon a coast-to-coast sampling 
of about 3,700 returns out of 9,300 blanks distributed. The present discussion should 
be viewed as an extension of the earlier report. 

* The ‘‘split-half’’ reliability of this attitude test is .94; of the information test, 
93. There is no short way of designating the validity of the entire attitude test, for 
that is a funetion of the separate ‘‘validities’’ of the component items and their 
interrelations, The statements retained in this measure were first marked by a 
‘‘eriterion’’ group of eminent liberals and conservatives in American public life and 
also ‘‘item-analyzed’’ in terms of the responses made by a national sampling of almost 
four thousand junior and senior high school teachers. For a detailed treatment of 
this difficult problem, see the writer’s ‘‘ Homogeneity of Opinion Among Liberal 
Leaders,’’? Public Opinion Quarterly, I, (1937), 75-78, and ‘‘The differential validity 
of items in a liberalism-conservatism test,’’ to be published in the Jowrnal of Social 
Psychology simultaneously with this paper. 
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jects of instruction have upon the beliefs of those who teach them as , 
means of livelihood. Since the statements to be judged deal mainly wit) 
problems in the political and economic spheres, the desirability of com. 
paring social-science teachers with all other non-social-science instructors 
is strongly indicated. The subject-matter that one teaches is presumably 
a symptom of one’s original preferences as well as an influential source 
of stimulation for one’s further development. Table I gives the necessary 
figures for a first test of this view. 


TABLE I 


COMPARATIVE Scores OF A SAMPLE OF SoOciAL-ScIENCE TEACHERS AND A SAMPLE or 
THEIR COLLEAGUES IN OTHER AREAS WITH RESPECT TO LIBERALISM-CONSERVATISM 
IN SOCIAL ATTITUDES AND FACTUAL INFORMATION ABOUT PUBLIC QUESTIONS 


Group Variable Mean Score SD N 

, , Attitude 69.78 14.84 70 
Social Science Teachers Information 57.54 15.80 * 10 
Attitude 67.90 14.15 163 


Wen-Geciah Science Teachers Information 39.60 18.20 163 

Since the higher the ‘‘attitude’’ score, the more ‘‘liberal’’ the group, 
it is clear that instructors who are most directly concerned with contro- 
versial problems are somewhat more likely than others to move away from 
the conservative position. However, the critical ratio, i.e., ‘‘reliability’’ 
of the difference between the two means, is only .93, so that it would be 
improper to emphasize this point. With respect to knowledge, of course, 
there is a more substantial and dependable difference, the critical ratio 
here being 5.22. There is a mild tendency for better information to ac- 
company greater liberalism—on independent grounds* we know that this 
ean be expressed by a correlation coefficient of approximately .40. Evi- 
dently those who have the ‘‘facts’’ about social issues are a trifle more 
likely to be progressively-minded with respect to possible solutions for 
such difficulties; or, if one wishes, a person with a liberal outlook is more 
strongly motivated to acquire and retain certain relevant data. 

In Table II are ordered the liberalism scores of instructors from va- 
rious fields picked at randon: out of a national sampling. It is a little hard 
to find an explanation for the sequence here revealed as in some instances 
the figures support empirically-based expectations and in others they fail 
to do so. Why art and music instructors—who presumably have much in 
common—should be at opposite ends of the seale is a bit difficult to under- 


* For confirming evidence, see the reference in footnote 1, especially pages 212-214. 
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TABLE II 


Soca, ATTrruDEsS Scores OF REPRESENTATIVE TEACHERS OF DIFFERENT SPECIALTIES 
(ARRANGED ACCORDING TO DESCENDING LIBERALISM) 


Subject Mean Score sD N 
Foreign Language 71.45 16.60 173 
Art 70.76 15.96 43 
Home Economics 69.12 9.76 68 
English 69.02 14.36 438 
Science 67.56 13.80 258 
Commercial Courses 67.04 13.76 184 
Mathematics 66.84 14.12 309 
Physical Education 65.04 13.92 67 
Vocational Education 64.84 14.60 112 
Music 61.56 15.28 35 


stand. Conceivably ‘‘radical’’ propaganda is currently operating more 
effectively in the content of the plastic arts (ef. Rivera, Orozco, ete.), or 
the isolated conservatory tradition of the musician is working to his par- 
ticular disadvantage in this sphere. The unexpected liberalism of home 
economies staff members may be a reflex of their nutrition studies and 
their recent rapprochement with social service workers. 


TABLE III 


LIBERALISM OF SECONDARY SCHOOL TEACHERS AS RELATED TO SEX AND 
Marirau Status 


Group Mean Score SD N 
Men 

Single 70.47 13.82 436 

Married 71.42 15.31 1162 

Widowed 78.00 —- 11 

Divorced 77.33 —- 12 
Women 

Single 67.42 13.38 1526 

Married 70.00 15.95 261 

Widowed 66.98 14.04 65 

Divorced 66.78 14.24 41 


Table III is of interest on at least two counts: first, its definite revela- 
tion of the greater liberalism of the male instructors, (perhaps due to 
their greater interest and familiarity* with political and economic mat- 
ters), and second, the superior liberalism of both married groups to the 
single folks, particularly in the ease of the women, where the critical ratio 
is 2.48 (with the men it is 1.27). It is known that marriage is sympto- 
matie of a favorable biological selection and the circumstance here dis- 


. The information median is 43.85 for the men and 36.95 for the women. 
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played may simply be another case of ‘‘good qualities going together.” 
Why being divorced or widowed should move men teachers in the direc. 
tion of greater liberalism and women instructors toward conservatism js 
one of the riddles of statistical inquiry that shows how little we know 
about the mechanisms involved in forming the spirit of tolerance. 


TABLE IV 


SociaL ATTITUDE Scores oF TEACHERS WHOSE FATHERS BELONG TO 
DIFFERENT ECONOMIC CLASSES 


Occupational Level Mean SD N 
Professional 69.13 13.88 371 
Small proprietors (includes farmers) 69.25 13.26 998 
Commercial 68.70 14.04 325 
Skilled labor 70.20 14.08 883 
Unskilled 68.70 13.21 113 


Table IV was developed as a possible test of the view that one’s eco- 
nomie antecedents condition one’s social outlook. However, so far as one 
ean tell from the telescoped data, there is a remarkable homogeneity of 
opinion among these teachers which seems in no wise related to the oc- 
cupational roles of their fathers. Perhaps these teachers have breathed a 
common educational and cultural atmosphere so long that the presump- 
tive initial differences have vanished. Instructors whose parents were in 
the ranks of unskilled labor—certainly near the bottom of the income 
ladder—are among the least liberal or most conservative of the social 
classes. At any rate, these figures give no support to the claim that early 
life in poor and underprivileged families (= ‘‘proletariat’’) creates a 
pronounced bent toward radicalism; indeed, the contrary is more likely 
to be the case. An oversimplified economic determinism appears to have 
misled many observers at this point. 

A further test of this type of economic determinism is presented in 
Table V. Presumably a teacher’s social sensitivity is intensified by the 
number of dependents—chiefly children—he has. Aid in the maintenance 
of necessitous relatives was a prominent feature of the depression and 
has probably always characterized the teaching profession with its rela- 
tively stable income. Nevertheless, the data fail to buttress this familiar 
assumption. A teacher with no dependents or with but one—say a wife 
or child—is as liberal as any of his associates who provide for bigger 
families. Progressivism is evidently not a function of large households! 

The final factor for which we have been able to accumulate evidence 
in this study concerns the effect of non-teaching occupational experience 
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TABLE V 


ATTITUDE MEANS OF TEACHERS WITH A VARYING NUMBER OF PERSONS 
DEPENDENT UPON THEM FOR SUPPORT 


Number of Dependents Mean SD N 
0 68.00 13.36 144 
1 69.35 13.27 78 
2 64.77 14.34 49 
3 63.74 12.79 29 
4 68.76 — 17 
5 68.60 — 5 


pon the index of liberalism. One of the questions on the personal data 
sheet asked, ‘‘ At what occupations other than teaching have you worked 
for at least six months?’’ Many experimental teacher-training programs 
exhibit a belief that contact with other (usually ‘‘humbler’’) occupations 
exercises a liberalizing influence upon the prospective teacher. It is as- 
sumed that having learned ‘‘how the other half lives’’ his sympathies and 
understandings will correspondingly be broadened. Although the kind 
of occupational experience is plainly of paramount importance, the “de- 
mands of quantification made it necessary to consider only the number 
of different non-teaching jobs held. 


TABLE VI 


SocraL ATTITUDE Scores OF TEACHERS WITH. VARYING AMOUNTS OF 
NON-TEACHING OCCUPATIONAL EXPERIENCE 


Number of Different Jobs 


Held at Least Six Months Mean SD N 
0 67.49 12.73 198 
1 71.71 15.27 73 
2 64.20 14.43 33 


The evidence of the averages on this point is none too clear, but if the 
increased variability is weighted it seems that a slight amount of non- 
teaching vocational activity may be an aid to the development of the 
liberal temper. The dose, however, must be small, for if teaching is to be 
the individual’s primary concern, he plainly cannot spend too much time 
profitably in other areas. Sponsors of new departures in the professional 
preparation of teachers will do well to keep this in mind. 

It would be a mistake to view the tabular material of this account as 
other than a rough preliminary clue to some of the factors which make 
one teacher a progressive and another a conservative. Teachers, like other 
citizens, differ widely in their opinions about mooted social, political, 
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economic, educational, and artistic matters. For a variety of reasons 
some cling contentedly to the old and others passionately pursue the ney. 
Some of these influences—perhaps exceedingly minor ones—have been 
studied here, and while all the preceding tables with their small differences 
need to be used cautiously, they contain acceptable evidence indicating 
that liberalism as an attitude in the field of human relations is most 
readily found among men teachers, among married teachers regardless 
of sex, and among instructors having few or no dependents. The sampling 
of teachers here employed is believed to be representative of the profes- 
sion in the United States today. Specific selective factors probably a 

count for some findings—e.g., the greater progressivism of the married 
teachers is not due to the effect of marital status as such, but to the fact 
that the abler and more alert teachers are somewhat more likely to b 
married. In general, our data do not so much show what experiences 


make for liberalism among teachers as what liberal teachers tend to do 
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EDITORIAL : 
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YEAR OF ACCOMPLISHMENT THROUGH COOPERATION 
AND COORDINATION 


\vr THE end of the year a well-managed business takes an inventory. 


Yo PURE IPL S AEP Aes 


ough few such oceasions since 1929 have been a signal for congratula- 
ns and rejoicing in business or educational circles, there seems to be 
optimism of the human spirit that will not be repressed permanently. 
\t the end of 1937 it is appropriate to take stock of educational ac- 
mplishments, especially in terms of codperative and codrdinated in- 
estigations. This statement is no reflection on the approximately three 
ousand graduate theses produced annually in education or on the large 
imber of other individual research projects worked out by professors 
ind field investigators. The independent worker has made large contribu- 
tions to education and will continue to do so, but the complexity and scope 
‘f many problems in the social sciences have increased to such an extent 
hat satisfactory solutions are to be found only in studies adequately sup- 
ported, staffed, and codrdinated. 

No previous year in educational history seems marked by so large a 
number of coéperative undertakings, in spite of the fact that earlier years 
ave ineluded large seale or national surveys or studies of secondary edu 
cation, teacher education, school finance, social studies, child health and 
proteetion, social trends, character education, modern foreign languages, 
Latin, reading, adult education, and genius. Much of the financial sup- 
port has come from the Carnegie Foundation for the Advancement of 
leaching, General Education Board, and the federal government. Among 
the coérdinating agencies are the American Council on Edueation, Na- 
tional Soeiety for the Study of Education, United States Office of Edu- 
cation, Research Division of the National Education Association, and the 
Social Science Research Council. The scope of the programs of investiga- 
tion mentioned below is suggested by the funds made available, in excess 
’ two million dollars for the five Office of Education projects, and in 

ertain other instances from a half million to a million dollars. 
The five major projects of the Office of Education are: forums, uni- 
ersity research, vocational education for Negroes, radio education, and 
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organization of local school units. The President’s Advisory Committee 
on Edueation made its report at the end of the year. The Social Science 
Research Council has published a series of thirteen research monographs 


concerned with as many social problems (including education) in relation 
to the depression. The monograph on education was prepared by the 
Educational Policies Commission of the National Education Association, 
which also sponsored Beard’s The Unique Function of Education in 
American Democracy. An American Youth Commission of the American 
Council on Edueation is in the midst of its five-year program of study 
The Committee on Revision of Standards of the North Central Associa- 
tion has published its findings in seven volumes. A Codperative Study 
of Secondary School Standards for the six regional accrediting agencies 
is near completion. The Committee on the Orientation of Secondary Edu- 
cation has presented to the Department of Secondary School Principals 
its two reports on the issues and functions of secondary education 
Another report of the International Examinations Inquiry appeared 
early in the vear in the form of Kandel’s Examinations and Their Sub- 
stitutes in the United States. 

Generous support has been given by the General Education Board to 
the work of the Progressive Education Association: Study of College 
Suecess, and the Commissions on the Relation of School and College, on 
Human Relations, and on the Secondary School Curriculum. More than 
one half of the states in 1937 were engaged on more or less extensive eur- 
riculum programs. During the year the Commission on the Social Studies 
of the American Historical Association completed publication of its six 
teen volumes. Although the American and Canadian Committees on 
Modern Languages from 1927-1931 published a total of seventeen mono 
graphs and a summary of the reports, unfinished and new projects 
required sponsorship, which has been and is being given by a committee 
on modern languages of the American Council on Education. The 
Regents’ Inquiry into the character and cost of public education in New 
York State is virtually completed, as is the nine-year Carnegie study of 
secondary and higher education in Pennsylvania. Of the 1937 yearbooks, 


space permits mention of only five: the National Society for the Study 
of Education on reading and on international understanding, the Depart 
ment of Superintendence (American Association of School Administra- 
tors) on the improvement of education, Department of Elementary 
School Principals on appraising the elementary school program, Depart- 
ment of Supervisors and Directors of Instruction on the changing ecur- 
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Ms 
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m, and the National Society of College Teachers of Education on 
se of background in the interpretation of educational issues. 

is suggested that the reader who wishes to keep in touch with such 
ets as these may do so through the various departments of the Jour- 
f Educational Research, especially through the section devoted to 
rch News and Communications. Merely to page through the num- 
of the Review of Educational Research, which has just completed its 
nth year, may prove a stimulating experience, although not all of the 
ture cited is research, and not all of the research summarized is good 
rch. It is possible that caustic and destructive criticism of research 
ngs and procedures, as well as fatalistic pessimism concerning the 


ire of education, has been overdone, even though the cauterizing effect 


have been beneficial. The evidence of the foregoing educational in- 


tory suggests to the worker in college or field that he face the new year 


a wholesome pride in his calling, with a reasonable confidence in the 


of evidence upon which the profession rests, and with a sincere ap- 
ition of and respect for the service rendered education through the 


re-scale codperative and coérdinated projects of recent years. 


CARTER V. Goop 
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tiers of the 


RHINE, JOSEPH B. New Fro 


ork: Farrar and Rine 
1937), 275 pp. $2.50. 

To describe the Duke Univers ty ex 
on telepathy and clairvoyance 


Rhine 
Mind. To one who 


ents 


idience Dr. has written 


Ve Frontiers of the 


es up t as his first reading in this 

eld el k may well seem to be an ex 
tremely g ind convincing populariza- 
tion of a large body of carefully per 
formed experimental work. The misty 
blue paper jacket with ‘‘mind’’ in smoky 
gr letters creates a striking first im 
ession of the book. The story is simply 
told and one’s interest is aroused and 
nt d by the rising drama of the 

Du liseoveries. But if one has read 
Rhine’s Extra-sensory Perception his 
‘detailed scientific report of the ESP 
res | ind the other publications 
the Duke laboratory, as well as the 

t ind check experiments of other 
nvestigators, one knows some of the de 
cts in the work: the lack of adequate 
ex ntal control, the gross over-esti- 
t f the statistical significance of 
some s s, the tendency to explain away 


yw scores, the contradictions and errors 
which appear in Rhine’s writings, the 
possibilities of clerical errors, sensory 
cues, and rational inference, and the pos 


unconscious dis 
subjects If 


difficult to 


sib ities f conseious or 
knows 


think of 


honesty n the one 


these defects, it is 


New Frontiers of the Mind without also 
thinking of them. But it is not necessary 
to review them all in detail here. In the 

e 
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lative to book reviews to Editor 


3OE 20E SE OE FOE BE 


» 
YY, 


ae 
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January, 1938 issue of the American J 
the Am 

Sociology are well docume 
reviews of the whole field by Wolfle 


Gulliksen respectively. These reviews offer 


nal of Psychiatry and 


Journal of 


letailed evidence in support of critic 
that can only be outlined here. 

Rhine devotes the first three chapt 
of New Frontiers of the Mind to givi: 
the setting for his experiments. The | 
“20 
beings, what are we? What is our plac 


The 


starts with a question: hur 


in nature?’’ recognized sciences, 


Rhine writes, have not given us the 
swer. He doubts if they ever can 
turns instead to search among the 
recognized and unusual fields. Work 


extra-sensory needless 
the field in 


Chapter IT relates a number 


perception 1s, 
say, which he hopes it wv 
appear. 
anecdotes on telepathy of the type wh 
every college teacher of psychology is 
customed to having introduced with t 


‘How do you explain this? 


question: 
In Chapter III the earlier work of Mes 
mer, Barrett : 
Richet, and Coover; the work of Gardner 
Murphy and G. H. Estabrooks under t 

direction of William McDougall, and U; 
Mental Radio 


(the Creery sisters case 


ton Sinclair’s are all 
viewed. 

Chapters IV, V, and VI are devot 
of Rhine’s ow 
Rhine 


how he, with his colleagues Lundholm ar 


to a narrative account 


work on clairvoyance. writes o! 
Zener, started out to investigate the al 
leged extra-sensory phenomena, of hov 
they first secured negative results, of th: 
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ment of the technique of asking 
jects to name, sight unseen, the 
m packs of cards containing five 
f each of five suits (plus sign, 
ectangle, star, wavy lines), of the 
ance of significant results when 
urds were used, of how Lundholm 
er withdrew ‘‘ because of pressure 
duties,’’ of the excitement when 
ject ran nine straight hits, of the 
excitement when he got 21 cor- 
it of a series of 25, and of the 
series of trials under varied con 
s with Linzmayer, Stuart, Pearce, 
e other star performers. 


pter VIl 


scusses the 


The First Serious Criti 
problem of evalu 
scores to determine whether 01 
ey can be explained on the basis 
ce success. Upon one point there 
general agreement. Even Stuart, 
‘’s assistant, is now agreeing with 
’s critics that his experimental con 
s violate the assumptions underly 
e probability theories which he has 
evaluate his data. However, a 
evaluation of Rhine’s data on 
ance would still show statistically 
‘ant secores—scores which might o1 
t not be due to clairvoyance but 
ertainly are not due to chance 
statistical errors in evaluating 
f telepathy were much more serious, 
me scores which he has called 
significant may well be due to 
e (cf. Kellogg, Scientific Vonthly, 
er, 1937). 
In Chapter VIII Rhine decides that 


ry cues from the cards could play no 


i in explaining his subject’s scores. 

ind, for example, that they did as 
i on new decks of cards as they did on 
; nes and so concluded that the cards 


ld not have been marked. Recently 


B. F. Skinner at Minnesota has made 


4 amazing discovery that the symbols 
4 
! 
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on cards supplied by Rhine to Dr. Baker 
(who was attempting to repeat at the 
University of Minnesota some of Rhine’s 
experiments) could be read from the 
backs. In printing the symbols, the ink 
has caused the cards to shrink, producing 
a slight impression on the back of eacl 
which corresponds in shape to the symbol 
on its face. If one looks squarely at the 
back of a ecard, or if one tries to look at 


a light through the card, the markings 
are invisible. But if light is reflected at 
the proper angle from the back of the 
eard, the depression can be seen and the 
eard can be read without difficulty. A 
newer pack of ESP cards, furnished with 
Zenith 


is much less plainly marked. I 


‘‘Compliments of your Radio 
dealer’’ 
made only 34 hits in my first 50 calls 
with these cards. This score, however, has 
the slightly significant ‘‘X’’ value of 
8.48. Under less favorable lighting con 
ditions, or with a subject who had not 
consciously recognized the existence of 
the marks but who was nevertheless oc 
casionally responding to them, scores sim 
ilar to those made by Rhine’s subjects 
could easly be duplicated. 

The only adequate way to ¢ xclude these 
cues is to use techniques in which the sul 
ject ean neither see nor touch the cards 
Rhine refers to some tests where these 
conditions were met 126-128) but he 
fails to give the total number of trials 
involved. 

When it is known that the cards are 
marked, some of Rhine’s statements pro 
vide interesting reading. In designing 
the cards, Rhine sought to ‘‘select forms 
as unlike as possible, even ih their parts’’ 
(48). The most common experimental 
technique allowed the subjects to ‘‘stare 
at the back of the card’’ (59). After these 
eards were developed ‘‘the tests for 
clairvoyance—began to give indications 


that the research was on the right track’’ 
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(62). In Eatra-sensory Perception we 


learn that the use of letters and figures 
on the cards was given up ‘‘ partly be 
cause it was quite laborious, and partly 


because of indications of failure’’ (35), 


and that ‘‘the best results were obtained 


with the 5-symbol cards’’ (53). In the 


directions supplied with the packs of 


1 


cards, subjects are told to ‘‘take the pack 


in one hand, carefully holding it face 


down and try to get an impression of 


which symbol you think it matches.’’ Un 
successful subjects are further instructed: 
four runs 


‘‘If your total seore for 


should not be above twenty, however, it 
would be better to try another test method 


or to try other conditions or another 
Again, in New Frontiers of the 
Mind, Rhine writes: ‘‘the cards 
which we are now using, and which have 
been made available to the general public, 
have been modified and improved from 
49). The improve 
ments seem to have consisted of simpli 
figures and printing them in 
bolder outline 

With each pack of cards comes a state 
ment that they are ‘*for the purpose of 


testing the until unrecognized 


re cently 


of the mind commonly called 


| owers 


’” 


intuitions, and mind-reading. 


Since the cards are marked on the backs, 
some of the tests described in the direc 
tions are of no value. In New Frontiers 
of the Mind, Rhine takes a different view 
of the lay use of his ecards. Such distribu 
tion, he writes, ‘‘does no one any harm, 


is commonly of good entertainment value, 


and . « (develops) powers of observa 
tion’’ (240 The markings on the cards 
rf « rse enhance these virtues. Under 
good conditions one can, as a stunt, call 
off all 25 correctly with ones eves so 


nearly closed that spectators will not 
open. But what will hap 


trick? Will 


know they are 


learns the 


pen when everyone 





[ Vol. 1, | 


interest in ESP decline as it did in 
jongg, miniature golf, and monopoly 
The work of other laboratories i 
viewed in chapter 1X. The only refer 
to the studies with negative results pul 
lished by Willoughby, Cox and Baker is 
the statement: ‘‘So far as I know 
three have failed to confirm (Rhine’s 
clusions), and those are the three series 
conducted with the least regard fo: 
Duke They 
under the direction of 


procedure. were not 
anyone with ex 
perience in this difficult field’’ (143 
four or five other places Rhine describes 
the technique as simple, easy, or ide 
suited for student research. 
Chapter X 


lepathy and devotes some half dozen pag 


discusses the work on 


to distinguishing between clairvoyar 


and telepathy. ‘‘It was not possible 


choose the symbols from anything 


jective or else we might have a basis 
interpretation 
(162). Thus the ag 


ecard 


an alternate possi 


clairvoyance’’ 


could not select a from the pack 
and look at it or write down the order 

which he intended to ‘‘send’’ the imags 
jut a hundred pages earlier (opposite 
60) Rhine 


trating a 


includes a photograph ill 
which t 
In the ¢ y 


tests 


telepathy test in 
sender is staring at a card. 

righted directions for telepathy 
which accompany the cards, the sender 

instructed to pick up the deck and co 
centrate on the first card. True, in bot 
eases the test is referred to as a ‘‘ tele; 
athy-eard test,’’ but if this name 

meant to distinguish it from an ordinar 
telepathy test, Rhine is merely addir 

confusion to his distinction between tele] 
athy and clairvoyance. 

The chance for clerical errors, mistakes 
and illusions is fairly large in the cor 
ditions under which the telepathic seri 
that tl 


points pe 


were run. Even Rhine found 


dropped five 


average score 
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25 when he had an independent 
heck the calls made by Zirkle, 
ng member of the Ownbey- 
n, the chief telepathic subjects. 
inclined to attribute this drop 


» the fact that Miss Ownbey had 
become Mrs. Zirkle (167 


less in Eatra-sensory Perce ption 


mmends the use of young mar 


les as among the best for study 
thy. 
nal six chapters discuss the gen 
tal setting of the work, the fact 
SP eannot be explained as a wave 
m since it does not obey the 
square law, the question of who 
sensory perception, the many 
has received from a variety of 
ents, the problem of time in 
nsory perception, and his plans 
finds a dilemma in the problem 
lelepathy, he writes may really 
yant pre-cognition of what the 
write down when he makes a 
f the symbol which he has been 


Conversely, clairvovance may ac 


be a telepathic pre-cognition of 
t which will be in the experi 
’s mind when he looks at the card 
the subject’s call. Just how to 
these difficulties, Rhine does not 
e does ‘‘feel that it is not 
to Say that this may be the 
, it 


t question science has ev 


the standpoint of an unbiased 


nemotional search for knowledge, 


dom of publish ng New Frontters 
Mind may be questioned. Aided by 


ution of Rhine’s patented ESP 


record pads, and by a popular 
ch has, for the most part, nr 
Rhine’s work at his own evalu 


; 
} 


has had the effect of selling 


nsory perception to an eager 


publie before the scientific world has been 
convinced of the soundness of the work. 
Less hurried popularization would have 
allowed time for other investigators to 
study the problem, for more thorough 
analysis of the questions involved, for 
criticisms to be offered and evaluated. If 
the work survived such analysis and 
criticism there would still be an oppor 
tunity for popularization; if it failed to 
survive, no scientist would want to pop 
ularize it, 
DAEL L. WOLFLI 


University of Chicago 


OGAN, R. W., Brxuer, L. E., CLARK, PAUI 
E., and Orners. A College Looks at Its 
The Spahr « 


Program Columbus: 


Glenn Company, 1937), pp. x 26. 

$3.00 

One of the most stimulating activities 
in which one may engage is a serious, 
critical analysis of his own activities. 
This is equally true of a group as well as 
an individual. Perhaps it is parti 


1 


ularly true of a group engaged in teach 
ing, whether at the « lementary, secondary, 
or college level. The stimulus to mprove 
ment which comes out of proper dissatis 
faction with what one is doing makes for 
progress. A critical analysis of one’s ac 
tivity is almost certain to result in some 
dissatisfaction. 

This volume records in part some 
the activities, experiences, and results « 
such a critical examination of the act 
ities of a group of earnest faculty mer 


bers in a small liberal arts college. V: 


rious phases of college life came under 
such critical scrutiny. Administration, 
curriculum, testing, teaching methods 
aims, and objectives are all included 


one or more reports in this volume. 
Some of the experiments are profes 
edly too limited in scope to yield gen 


eralizations. They were not too limited in 
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mental studies (geology, Bible, chen 
and history), testing (voice improve 
English composition, and musical tal 
ol J} etives social seit nce, biology, 5} 
and English), curriculum super 
preparation of elementary teachers 
composition character education, 
administration (job analysis of the 
of dean of women, residence dist 


tion of students, marking, and social] 


While the studies are of unequal 
in terms of objective evidence, scie 
procedure, and application to situat 
in other higher institutions, the prog 
of investigation is more than justified 
terms of solution of local problems 
faculty improvement. Publication of 
studies is decidedly worthwhile, poss 

it so much for the actual investigat 
results reported as for the service 
dered through a concrete illustration 
erative program for faculty st 
lation, in-service training, and impr 
ment. How much better it is to ad 


uch plan than to follow traditior 


yr modifications on t 





assumptions of vari 


staff members, or to accept and advert 


basis 


] 


without real evaluation some promisi! 


innovation as the solution of all the d 
culties of the liberal arts college. 
CARTER V. GOO! 


Unive ty of Cincinnati 


WILLIAMS, JESSE FeErrRInc and SHA 
FANNIE B. Methods and Materials 
Hlealth Education New York: Thor 
Nelson and Sons, 1935), pp. VIII 


331. $ 1.65. 


The literature in the field of hea 
education has lagged considerably beh 
the development of the field itself. 


this book by Dr. Williams and Miss Shi 


the educational world is presented wit! 
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gh-going and competent discourse 
he various phases of school health 
which should be of considerable 
| assistance to those administra 
1 teachers who feel the need of 


in the development of their pro 


pe of the book is wide and its 

ts understandable and helpful. To 
vith the authors have wisely de 
s and from the definitions sug 
which, incidentally, have been 
at cepted by the American Physi 
ation Association) the reader can 
ear picture of the various admin 
e sub-divisions such as health ser 
i health instruction. At the same 
will be informed of the fact that 
rms as ‘‘ medical inspection ’’ and 
hygiene have outgrown their 
ess. This chapter on definitions 


to clarify terminology and thus 


~ 


the educator with a language which 


and clear. 


that introductory chapter the text 


hes at once the nature of the child 


act, concise, and thorough anal 


f child life and the influences which 


nit. If today’s education begins 
hild then every teacher should 
of the progress if his de velo} 
his biological growth, the relation 
rganic parts, the nature of dis 
he factors influencing de 

nt, and, the emotional nature of 
rganism. These matters are s¢ 
th clarity and understanding and 


ider has the confidence that the 


al is scientific. 


chool influences the child not 


the classroom but continuously 


ghout the day. Its influence, fur 


is more than mind-affecting 
i est ealities are commonplace to 


w but in these succeeding chap 


e authors have shown us how these 


influences can be organized, how they can 
be administered so that the relation of 
school life to the health of the child can 
become more than a merely happenstance 
thing. Teachers want suggestions on cur- 
riculum construction, or testing and meas 
uring, and principally, perhaps, on ma 
terial for classroom use. The discussion of 
these matters in this book are sound, and 
useful, and presented with authority and 
wisdom, Other chapters on the role of 
official and non-official health agencies, 
and on the health of the teacher com 
plete the book. 

The merit of the book is a reflection, 
of course, of the professional skill of its 
writers. The senior author, Dr. J. F. Wil- 
liams, is Professor of Physical Education 
at Teachers College of Columbia Unive 
sity and is so well established in the joint 
area of physical education and healt! 
education as to be rightfully considered 
eminent as an authority on the health 
of the school child. Miss Shaw, the ju 
thorough experience as a field worker and 
teacher. She is now attached to the staff 
of the National Tuberculosis Association 
as a field worker in health education. 

The book will prove ust ful for botl 
teachers in the field and in training. The 
chapters have outlines at the beginni 
and review questions at the end. The 


ndex is complete and in all respects thi 


i 
book is to be recommended as a definit 
contribution to our knowledge of educ 
" ma its processes 


Ohio State University 


tis Relat oO to t) ( ( 
tributions to Education. No. 667 (New 
York Bureau of Publications, 7 I 
ers College Columbia University 
1935), pp. xi + 99. $1.60. 








Extra-curricular 


JOURN 


activities, 


as defined 


by the author of this book, are negatively 


characterized 


not 


mo cre 


seribed 


by three 


1 the regular 


Ji? 


however, 


regular schedule, and that a ‘* 


t « 


outline of 


a 


th 


pr rT 


it 


te 


course of 


the 


onsidere d by 


‘edu! 


a course of study. 


limitation 


Ss on tl e 


and his use 


+} 


ricular activities for him 
found in the refusal to g 
leading to graduation. 


stu 


dy 


Ni 


essential 


yn-credit 


in 


characteristic 


might, 


little 


hookkeep 


ely 


heen 


schedule ; 


Ia ad 


r 


ttion ;" 


traits. They are 


they receive 


they are 


yt carried on in accordance with a pre 


study. We must note, 


him as 


activity 


a 


period’’ is 
part of the 


suggestive 


es’’ is not regarded as 


In view of the author’s 


first and 


oO 


Sé 


ve 


third traits 


italics for the second trait, 


f extra-cur 
ems to be 


them credit 


therefore, better be 


Activities in rela 


m to Activities Which Receive Credit 
oward Graduatior The term curricu 
m may then be used, as the author 
nks it should be used, to inelude ‘‘all 
e activities and experiences provided 
the school * Thus the ambiguous 
rm extra-c cula will be avoided. 
final sentence in the book when r 
ated in these t precise terms he 
mes The pr ressive ser da y 
| 1 uses the whole life of the schoo 
rT edu it e pur} ses and s une 
ed whether the desirable outcomes at 
e results of activities which receiv 
lit toward graduation or of non-credit 
tivition 
To the reviewer this whole attempt to 


guish between credit and non-credit 


importance, The use 


mdary 


ing device 


’ 
allowes 


to 


ucation 1s 
which has 


determine 
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the whole philosophy of education 

level. The essential characteristic 
so-called extra-curricular activitic 
be found in the freedom which e¢ 
should be allowed to the pupils 

itiating, planning, executing, and 
ating these activities. In the regu 
tivities in the classroom the prim: 
sponsibility must rest with the te 
but in clubs, assembly, athletics, 
cations, musical organizations, dr: 
organizations, and student coun 
primary responsibility should rest 
the pupils. 


For those who may be concerne 


the question of credits for student 


tivities outside the regular act 
of the elassroom this book g 

well-written report of practices in 
Briefly summarized, the trend is t 
credit toward graduation for work « 
newspaper, for membership in musi 
ganizations, and for participation in 
maties and debating. On the other 

athleties, assemblies, clubs, and st 
eouncil will apparently remain on a 
redit basis.‘ 

With the exception of Table VII 
headings of the tables are clear and 
quate. The statistical treatment of 
data makes effective use of the e1 
ratio technique. In Table V an err 
fact occurs. One school reports that 
ketball was introduced before 1890, 
Dr. James A. 


game in 1892. In this same table an 


Naismith invented 


of computation is found. The first 


tile vear for introduction of the m: 


should be 1902 instead of 1900.5 


Superintendent of Ne hools, 


Shaker Heights, Cleveland, Ohio 
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BULLETINS 


Saun. ‘*‘Are the New York 
ls Retarding Mental Development 
Democracy?’’ (Flushing, N. Y.: 
t Press, 1937), 13 pp. 
uthor emphasizes the fact that the 

help the growth and develop 

emocracy if the school’s aim is 
il in character and provided that 


ill phases of education. 


i 


WitutiAM G. ‘‘Improving Social 

St es Instruction,’’ Research Bulletin 

Vational Education Association, 

Washington: National Education 
ation, November, 1937), 69 pp. 

based upon 1764 replies to a ques 

submitted to teachers of social 

S$ 1s com] iled and diseussed on the 

bjectives, content of the pro 


ds and aids in teaching, deal 
ontroversial subjects, and the 
inity as an area of study. Bib 
ies of standard tests, texts and 


mentary books are given. 


DENNIS H. ‘*‘How Do Teachers 
Themselves ?’’ Re} mt from Sej 
: Educational Administra 

d Supervision, 4 pp. 


ithor studied the self-ratings of 
hers as compared with principal ’s 
s, the Coxe-Orleans Prognosis Test 
ching Ability, and an Aptitude 
Teaching. He finds few signifi 


relations, 


JONES, J. W.., 


State Teachers College, from Au, 


to June 1937 
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os Address all communications relative to research abstracts and 
ve bibliographies to A. S. Barr, University of Wisconsin, Madison, %& 
Wisconsin. 
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Hoty, T. C. ‘* The Disposition of School 


Bond Issues and Special Levies Sub 
mitted in Ohio School Districts Between 
January 1 and November 2, 1937, hl 
clusive,’’ Bulletin of the Bureau of 
Educational Research (Columbus: Ohio 
State University, November, 1937), 23 
pp. Mimeographed. 


This bulletin tabulates the disposition 


if school bond issues and special levies 
‘or both those submitted in Ohio city an 


Ohio county school districts. 


; Editor. ‘‘Abstracts of 
Unpublished Masters’ Theses,’’ The 
Teachers College Journal, VIII (Terre 
Haute: Indiana State Teachers C: 
lege, i937, 37 pp- 


This bulletin supplies abstracts of un 


published Masters’ theses at the Indiana 


Further Study of Certain Techniques 
Used to Individualize College Teach 
ing in a Selected Group of Land-Grant 
Institutions,’’ Bulletin of the United 
States Depa tment of Aart ly e, Ex 
Washington: United 


States Department of \ 


tensio Sé 


1937), 54 pp. 


The purpose of this study is to show 
what a g p of agricultural colleges 
are doing to provide certain kinds of sp 
cial treatments to meet individual differ 


nees of students. Bibliography 
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Woops MARY TENISON. ‘‘ Juvenile De 
linquency,’’ Bulletin of the Australian 
Coun il for Ed catro al Re search ( Mel- 
Melbourne University Press, 

1937 , a Py 

The author discusses the general prob 
lem of delinquency and then the existing 
Australia with recommenda 


tions for after-care and prevention of de 


BARBARA H. ‘* When 
Leave School.’’ Bulletin of the Division 
of Ing Public 


Schools Division of In- 


Pupils 


truct on, V Le tpolis 
Minneapolis: 
struction, October, 1937), 16 pp. 


} 


This bulletin contains reports on high 


" 


graduates a year after graduation 


and on pupils finishing junior high school 


YUCATIONAL ADMINISTRATION AND 
PERVISION 

AND WEBB, JAMES F. 
‘Cumulative Records for Elementary 
Schools,’’ The Elementa 


val, XXXVIII (December, 1937), pp. 


FLoRY, CHARLES D., 


4 School Jour- 


278-290 
( ilative record forms are given as 
ggestions to administrators who are 
trying to evolve satisfactory systems of 
rl 


An Investigation of 


the Length of the Elementary School 

Dav,’’ The Elementary School Journal 

XXXVIILT (December, 1937), pp. 291 

mas 

On the basis of data gathered by ques 
{ nnaire trom ¢ ghty elementary schools, 
the ithor forms a typical school day for 

t Is 


Reavis, G. H. ‘*The New Ohio High 
School Standards,’’ Educational Re 


, XVI (Columbus: Ohio 


earch B et 
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State University, December §8, 


oon 90° 
de 


PP. <<0-= 
A syllabus discussing the new 


school standards for the state of 


METHODS OF TEACHING AND LEAR) 


Davis, H. Curtis. ‘‘Improving R 
Ability of High School Seniors,’’ ¢ 
fornia Journal of Ne condary Educ 
XII (November, 1937), pp. 424-4 
The author reports an investigati 

improving the reading of high s 


1 


seniors through instruction, and finds 
the gain in comprehension of tl 
structed group represents a two-year 


for the average child. 


HILLIARD, GEORGI H., AND TROY 
‘<Tnformational Backgr 

hg Readiness 
Reading The Eleme 


School Journal, XXXVIII (Decer 


1937), pp. 255-265, 


ELEANOR. 
as a Factor in Readin 


Progress, 


On the basis of data secured about 
kindergarten children, who were stu 
through grade two, the authors con 


that, other factors beiz equal, child 


with rich backgrounds are more stro 
equipped to attack the printed page t 
are pupils of meager backgrounds. 


CURRICULUM 


CowLey, E. B **Ratio and Proport 
in High School Curriculums,’’ S¢ 


Vathematics, XXX\ 


1079-1088, 


Scvence and 
(December, 1937), pp. 
The purpose of this article is to st 
the high school pupils’ grasp of 

tionship of measurable quantities as 
vealed by the understanding of ratio 
proportion. The author examined mat 
maties, physics and chemistry texts; ¢ 
cles by teachers in the field; and aq 


tionnaire sent to pupils, to determine 
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e of this concept in the high 


irriculum., 


Puiu. ‘*‘Student Labora- 
Work Versus Teacher Demonstra- 
s a Means of Developing Labora- 
Resourcefulness,’’ Science Educa- 


XXI (November, 1937), pp. 185- 


uuthor attempted to find out 

allowing students to do labora- 

rk is a better way for them to 

laboratory resourcefulness than is 

emeonstration, and concludes that 
is the best. 


CHILD DEVELOPMENT 


Joun G. ‘*Scholastie Achieve- 
nd Measured Maladjustment,’’ 
Journal of Applied Psychology, 


‘XI (October, 1937), pp. 485-493. 


iuthor coneludes that measured 
stment and radicalism may de 

evement below the expected 
ess it is affected by some counte1 


int 


KEMENT, STATISTICS, AND 
CIENTIFIC TECHNIQUES 


Minton A. ‘fA Comparative 
ly of Seale Constructed by Three 
physical Methods,’’ Psycho 


Septe mber, 1937), pp- 


arison is made between the 
s constructed by the Method of 
| Comparison, Rank Order, and the 


pt 
f Successive Intervals. 


Epira C,. ‘*‘A Project for the 
ation of School Broadcasts,’’ Ed 
4 onal Research Bulletin, XVI (Co 


is: Ohio State University, Decem 


8, 1937), pp. 238-9. 


The author discusses the project for the 
evaluation of school broadcasts, now being 


carried on at Ohio State University. 


NEW TESTS 


Bayrorr, A. G. ‘‘Quizzes on Dashiell’s 
Fundamentals of General Psychology,’’ 
Chicago: Houghton Mifflin Company, 
1937. 

This battery which consists of twenty- 
two objective tests, one on each chapter 
of Dashiell’s Fundamentals of General 
Psychology. The test aims to provide a 
comprehensive measure of the student’s 


knowledge of the subject. 


DURRELL, DONALD D. ‘‘ Durrell Analysis 
of Reading Difficulty,’’ Yonkers-on- 
Hudson: World Book Company, 1937. 
A battery of individual tests in oral and 

silent reading, supplemented by a check 

list for observation and record of faulty 
habits and inadequate skills in the differ 
ent reading abilities. Suitable for grades 

1-6. 

DURRELL, DONALD D., AND SULLIVAN, 

HELEN Buare. ‘‘ Durrell-Sullivan Read 

ing Capacity and Achievement Tests,’’ 

Yonkers-on-Hudson: World Book Com 

pany, 1937. 

The Reading Capacity Test is a group 
test for measuring the child’s capacity to 
learn to read in terms of his ability to 
understand spoken language and is con 
posed entirely of pictures. The Achieve 
ment Test is a group test which may be 
used mdependently to give an analytical 
measure of reading achievement. Both 


are suitable for grades three to six. 


ENGEL, ANNA M., AND BAKER, HARRY J. 
‘Detroit Beginning First-Grade Intel 


ligence Test,’’ 


Y onkers-on-Hudson: 
World Book Company, 1937. 
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This is a revision of the Detroit First 


G le Intelligence Test, and is built upon 
the ise of the earlier test and on re- 
search witl Detroit first-graders. 
he manual for this non-reading test pro 
les reé rms, gives careful dire 
tions r interpreting, and shows that the 


r the test has been increased. 


Met L, W iM A., AND HER 


RING, JOHN 


r. **A ¢ prehensive Test Program,’’ 

( cag Laidlaw Brothers, 1937 
Ls rour tests « sisting of ' 
Inte gence Test in Educational Back 
Questionna which aims to 
e as} ts f ed if lity not meas 
by intelligence tests; a Comprehen 
ea t Test t easure desir 
e | nings nd a Practices 
Question ; evaluates achool 
tices The itt s suitable ) 
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grades from four through nin 


norms are given for each test. 


‘*Study Exercises in Reading and 
metic for Grades 5 to 8,’’ Bloomi: 
Publie Publishing Cor 


1937. 


School 


The aim of these study exercises 
ind those opportunities by whi 


thinking power of the pupil is iner 


i 
the exercises are presentations of 
experiences and discussions away 
school situations, 

TYLER, HENRY T., AND KIMBER, GEO! 


} 


‘*Tvler-Kimber Study 


Stanford University: Stanford 
versity Press, 1937. 
The test provides eight separate 


ures of skills and unde rstandings of 


techniques, and is suitable for use in 


seh ol and college. 


Skills Ti 
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Address all research news and communications to Carter V. 
jood, Teachers College, University of Cincinnati, Cincinnati, 3 
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OCTORS’ THESES UNDER WAY IN EDUCATION, 1937-1938 


CARTER V. 


Goop 


University of Cincinnati 


s the eighth annual list of Doe 
ses under way in education to be 
1 in the January number of the 

Educational Research. The 
st includes only the titles which 
t been printed in the January, 
1936, and 1937 numbers of the 
Therefore, in order to make a 
y complete canvass of doctoral 
under way at present, it is neces 
examine these four lists, 1935-38. 
ted dissertations may be dis 


through eanvassing the annual 


phies of research in education, 


a] 
ed by the United States Office of 
tion, and through Doctoral Disse 

lecepted by American U1 

blished annually by the H. W. 
n ¢ As reported in the latter pub 
Doctors’ 


iwerst 


degrees 


n, the number of 
m awarded during 1935-36 was 
uring 1936-37 was 290. The 

| theses for all fields during 1936 

ed 2709. The number of educa 

titles reported in the Wilson pub 
s rather consistently smaller than 

Office of Education annual bibliog 
esearch; the latter has listed 

ition nearly 400 doctoral disse: 

annually for a number of year 
vear certain of the titles cause 
marvel at the breadth of the field 
ation, as it is interpreted by at 


some institutions, for example: 


Co-defendants 


‘The Effects of Acid-and-Base Forming 
Diets on the Calcium, Phosphorus, and 
Nitrogen Metabolism of Children,’’ ‘‘ Re 
Illinois, ’’ 
Literary Critic,’’ 
Validity of 


Study of 


gional Study of Southern 
‘* Horace Walpole as 
‘A Study of the 
Cardio-Vaseular Tests,’’ ‘‘A 
in Major 
Belaseo,’’ and ‘*C¢ 
Altitude 
Home Ee 


some 


Crimes,’’ 
** David rrelation of 
Sugar Proportion and 
With Applications in 


Education 


Levels 


‘onomics 


name of the student, 


1 8es8 t} e 


thesis title, and graduate institution are 


iven, and in most instances the major 


adviser or advisory committee is listed. 


No differentiation is made in this com 


pilation between Ph.D. and Ed.D. prob 


The institutions which have sent titles 
for inelusior the 1937-38 list are: 


rsity, Arizona, toston 


College, Boston University, Bryn Mawr, 
Buffal California, Cathol University, 
Chicag Cineinnati, Clark Colorado, 


e College of Education, Co 
lumbia, Cornell, Duke, Fordham, George 
Illinois, 


Colorado Stat 
Harvard, Indiana, 
Johns Hopkins, Kansas, Kentucky, 
Marquette, Michigan, Minne 
sota, Missouri, Nebraska, New York, 
North Carolina, North Dakota, North 
Oklahoma, Oregon, Pea 


Louisiana, 


western, O) wo, 


body, Pennsylvania, Penn State, Pitts 
burgh, Rutgers, Southern California, 
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Stanford, Syracuse, Texas, 


Temple, 


gr , Washington, Western Reserve, 
consin, and Yale. 

After a lapse « f several years and at 
the request of users of this compilation, 


a topical index of the titles again has 
An effort has been made 


title 


been prepared. 
each 
headings; for example, ‘‘A 
History of Negro Land-Grant Colleges 

is vogrel under both ‘‘ Higher 

‘History of Education. 
the index refers to the 
the alphabetical 


to classify under one or more 


ay pro} riate 


wenn 
The 


cor 


tion’’ an 
numeral in 
responding number in 


list of candidates. 


As in 


art, toreign 


other years, such special fields as 


languages, home economics, 


ind music have been studied relatively 
infrequently in doctoral theses. Those 
areas receiving most attention are easily 
apparent, on examination of the index. 


Iimiunistratton and S iperviston. 
1, 2, 3, 5, 7, 10, 11, 15, 16, 18, 25, 28, 
29. 34. 38, 41, 48, 49, 51, 56, 60, 66, 69, 


100, 106, 112, 


~ 2. see 87, S88, 89, 91, 


113, 121, 125, 127 , 140, 150, 158, 160, 
162, 165, 167, 168, 169, 170, 176, 178, 189, 
190, 193, 199, 200, 202, 205, 2 211, 214, 
216, 220, 2 994. 297, 229. 232, 235, 246, 
24/,2 958. 264, 265, 266, 267, 270, 277, 
81, 282, 292, 294, 29 01, 303, 305, 307 
20, 326, 327 15 16, 347, 348 19. 360 
61] 65, 375 76, 385, 387, 389, 390, 392, 


», 422, 425, 429, 433 
, 461, 


Psychology and Measurement (Gen 
il t. 6, 7, 8, 9, 12, 14, 15, 21, 36, 37, 


111, 
161, 
224, 


286, 


+0, } . 15, wo, 54, 58, oy, das SU, Sl. 
120, 130, 131, 187, 143, 145, 156, 159, 


D, 195, 210, 213, 217, 


8, 309, 312, 318, 319, 326, 328, 340, 341, 


22°90 ef 2 _—o 


542, 344, 350, 351, 362, 363, 365, 369, 373 
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84, 394, 396, 409, 410, 424, 443, 44 
155, 464. 

General Method and Theory, |} 
tional Philosophy, and Sociology.—1 
46, 74, 77, 86, 87, 92, 9 
119, 123, 129, 135, 137, 
181, 187, 192, 196, 197, 


022 oF 
ww, wt, Ded, 


101, 108, 116, 
142, 152, 177, 


256, 259, 263 


234, 240, 248, 250, 255, 
287, 296, 308, 311, 318, 353, 380, 386, 


£13, 423. 
History of Education and oy 
15, 22, 31, 44, 61, 


’ 


Education. 


624 


t 


69, 72, 74, 81, 91, 95, 96, 103, 104, 
108, 109, 126, 130, 140, 142, 147, 151 
155, 171, 172, 190, "192, 196, 203, 218 
226, 234, 235, 246, 252, 254, 270, 271, 2 


»- 


274, 302, 304, 308, 311, 
377, 379, 381, 393, 400, 
$21, , 432, 444, 448, 

Preschool and El 


20, 344, 353, 
£05, 411, 412 


£28 $59, 467. 


102, 
181, 
328, 332, 343, ; 
401, 404, 
161, 465. 


Seconda 


103, 112, 117, 


183, 186, 204, 


119, 132, 134, 
219, 251, 293 


, 015 
151, 356, 362. : 
106, 415, 418, 428, 432, 


£39, 


y Education. . J. 


24,26, 29, 39, 44, 51, 62, 63, 76, 84, 


128, 133, 1 


88, 106, 110, 112, 115, 117, 


162, 166, 179, 188, 189, 205, 224, 232, 
240, 244, 248, 252, 256, 258, 267, 268, 
271, 275, 377. wie 288, 293, 296, 297 
306, 309, 314, 323, 342, 347, 357, 359, 
371, 379, 382, 391. 395, 414, 421, 42 
$29, 434, 436, 447, 454, 459, 466. 


Hliagher Edu Teacher 


cation, 


72, 78, 90, 91, 93, 100, 107, 110 


ementary Educat 
5, 12, 14, 37, 68, 80, 82, 83, 89, 98, 


148, | 


: 


Traini 
and idult Education. ll, 15, 1 @ 19, om 
30, 31, 33, 35, 36, 42, 43, 54, 55, 60, 


112, ] 


f 


9, 390, 39 


3, 4: 


117, 122, 125, 126, 134, 136, 138, 149, ] 
157, 159, 162, 163, 165, 174, 175, 184, 18 
202, 203, 205, 209, 222, 227, 228, 238, 


oC 


242, 243, 247, 249, 253, 262, 270, 276, 
85, 286, 291, 292, 294, 298, 299, 303, 3/ 
05, 317, 319, 322, 324, . dal, aaa 
36, 337, 358, 339, 348, 355, 356, 358 
68, 372 i4 75, 378, 380, 382, 383 
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37, 391, 402, 416, 417, 419, 427, 430, 
7. 453, 455, 460, 462, 463, 467, 468. 
131, 271, 306, 343, 411, 426. 

acter and Religious Education. 
2, 83, 109, 111, 124, 146, 147, 174, 
54, 259, 263, 269, 276, 285, 287, 302, 
367, 370, 395, 400, 407, 412, 420, 
145, 453. 

cial and Business Education. 
_ 84, 262, 279, 284, 291, 314, 339, 
87, 427, 429, 457. 


sh, Reading, and Language.—39, 


8, 132, 145, 148, 156, 171, 172, 186, 


201, 203, 204, 206, 230, 232, 238, 241, 
999, 310, 312, 316, 324, 325, 330, 333, 
54, 357, 377, 379, 388, 403, 404, 406, 
131. 460, 465, 466. 

qn Languages.—53, 144, 166, 225, 


17. 73 07. 975. 399 


Economics. 


oS, 398. 
al, Industrial, Vocational, and 
Itural Arts—20, 57, 78, 94, 104, 
150, 182, 208, 209, 233, 255, 295, 306, 
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